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Abstract—This paper outlines some design requirements for situation description, policy 
specification and decision consequences feedback systems of the type that could be used 
to help manage RDT&E portfolios for large organizations. Seven of these Decision 
Support Systems (DSS) were inspired by a DSS subsystem classification scheme 
developed by Alter. Recently developed multipurpose software packages of an inter- 
active, user-friendly nature are also recommended. Three additional systems design 
characteristics are proposed: a convenient multi-input format for policy guidelines and 
constraints; an organizational incentives index for evaluating proposals; and a historical 
analogies subsystem based upon the company’s previous experience with decisions of 
a similar nature. 


INTRODUCTION 


The term “Decision Support Systems” (DSS), while relatively new, is suitable for our 
present purpose, of specifying the design requirements for some important situation descrip- 
tion, policy specification, and interactive decision consequence feedback systems of the type 
that could be used by those who manage Research, Development, Test, and Evaluation 
(RDT&E) portfolios for large organizations. 

A previous publication by the authors[1], outlined plans for an Investment Strategy 
Decision System (ISDS) for RDT&E. 

In our 1981 paper, an ISDS 1s conceived as: 


‘...a sequence of procedures concerned with: (a) the preliminary screening of RDT&E 
proposals (regardless of where they originate) by line supervisors and technical directors; (b) 
the comparative evaluation of proposals with each other and with ongoing efforts; (c) the 
aggregation of surviving proposals and ongoing efforts into organizational investment packages, 
each of which is assembled in accordance with program objectives and budget and policy 
constraints; (d) the comparison of alternative multi-year investment packages with each other; 
(e) the display of alternatives that would show managers how various decisions they might make 
are expected to impact program accomplishments as well as budget constraints; (f) the display 
of summary data regarding predicted trends from year to year; and (g) the display of statistics 
required for reports...” [1]. 


The present paper expands the thoughts presented in this previous paper by describing 
the kind of DSS subsystems that are required to make an ISDS work. 

Although there is considerable management interest in investment strategy decision 
systems|2] and many systems have been developed, most of these systems have not been 
put to use. The lack of acceptable systems is especially evident in portfolio management 
systems for RDT&E. The inputs for RDT&E programs are extremely complex, the 
outcomes are uncertain, and circumstances change rapidly. The uncertainty is so great that 
mathematically oriented, fixed-policy, fixed-input systems are usually overtaken by events 
so fast that nobody has a chance to use them [3]. This susceptibility to change 1s especially 

+This paper was not prepared as one of Dr. Robert Stephenson’s duties at the Air Force Human Resources 
Laboratory. The opinions expressed in this paper are the authors’ own, and do not necessarily reflect an official 
position of the U.S. Air Force. 
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true of long range plans for RDT&E. A fifty percent annual turnover in long range plans 
of the “‘soft’’ variety is not at all unusual. 


DECISION SUPPORT SYSTEMS 


In agreement with KEEN and MorTON|{4], Decision Support Systems (DSS) are defined 
as management information systems that: (1) assist managers in their decision processes in 
unstructured tasks; (2) support, rather than replace, managerial judgment; and (3) improve 
the effectiveness of decision making rather than its efficiency. As might be expected under 
circumstances which are relatively unstructured, the emphasis is upon building systems for 
specific decision situations and decision makers, so there is no “‘typical” design. 

Guidance regarding the types of subsystems that could be incorporated into a DSS is 
provided by ALTER[5], who categorized 56 DSS subsystems into seven reasonably distinct 
types: (1) “file drawer” systems, which allow immediate access to data items; (2) data 
analysis subsystems’, which allow for the manipulation of data; (3) analysis information 
subsystems, which provide access to a series of data bases and small subsystem models; (4) 
accounting models, which estimate the consequences of actions on the basis of accounting 
definitions; (5) representational models, which estimate the consequences of actions on the 
basis of simulated events and conditions; (6) optimization models, which provide guidelines 
for action by identifying and/or generating high payoff solutions that are consistent with a 
series of constraints; and (7) suggestion models, which identify a variety of decisions that 
would improve the desired outcomes[5] p. 74). 


DSS subsystems 

The design requirements for these seven DSS subsystems follow. The function of 
each subsystem in an ISDS for RDT&E and an example of its output are provided in 
Table 1. 

(1) “File drawer” subsystem. The file drawer subsystem would display inputs and 
previously computed outputs for user review. It is not much to ask that stored information 
be displayed when it is requested, but this simple requirement can get complicated when 
many different kinds of information are provided and the information is organized in 
different ways. 

(2) Data analysis subsystem. Routine data manipulations and computations are not 
very glamorous either—but these routine computational capabilities get complicated when 
there are many different ways (e.g. fiscal year, product, cost category, benefit category) in 
which long range plans can be computed, restructured and reorganized. 

(3) Analysis information subsystems. The DSS components become more interesting 
when we get into analysis information subsystems, which differ from the simple data 
analysis subsystems in that external inputs and data processing models are needed. The 
number of special inputs and subordinate models that are required is an index of the 
complexity of the total system. Typical inputs and subordinate models for analysis 
information subsystems for RDT&E decisions are: (a) formulae for increasing or 
decreasing resource requirements and/or recomputing the probability of success when the 
product delivery schedule is changed; (b) procedures for identifying the critical path in a 
task performance network; and (c) formulae for estimating the probability of subsequent 
outcomes based upon changes in the estimated probabilities that the preceding events will 
actually occur. 

(4) Accounting models. Many accounting transformations of a routine nature have 
already been developed for use in management information systems. Examples are: inflated 
value computations; discounted dollar computations; and procedures for converting man 
years into dollars. Many of these models and their accompanying data bases can simply 
be incorporated into a DSS for RDT&E management with little or no change. Other 
accounting models could conceivably require special processing before they would be 
incorporated into decision support systems for an ISDS. 

(5) Representational models. Most representational models in RDT&E are based upon 
network-based project planning, scheduling, control and cost estimation procedures. The 
literature has many network models of this type, the best known being the Program 
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Evaluation and Review Technique (PERT) and Critical Path Method (CPM) and their 
modern day applications: the shortest route problem, the maximal flow problem, and the 
minimal spanning tree problem. Good reviews of these techniques can be found in 
WAGNER[6] and LoomsBa[7}]. An example of how they can be incorporated into DSS 
systems for RDT&E decisions can be found in the recent book by THIERAUF[8]. 

(6) Optimization models. Optimization models can either be fixed, depending upon the 
inputs provided by the analyst, or interactive, in which case the conditions of optimality 
would be redefined by managers as they work with the system. In the case of an RDT&E 
DSS, the best strategy is to establish an interactive system, so that managers are able to 
restructure optimality criteria and look at the consequences of changing the weights 
assigned to each component. 

(7) Suggestion models. Suggestion models are designed to confront the decision makers 
with alternatives that they would not otherwise have considered. Some of these alternatives 
may have been precluded by the constraints that the decision makers imposed before the 
analysis was conducted. If, for example, the decision makers required that a certain 
percentage of the RDT&E be devoted to product X, and product X is expected to result 
in less profits than those that could have been provided if some other product were 
supported, the decision makers can be confronted with the logical consequences of their 
position. Other recommendations generated by suggestion models might require changes 
in the budgeting and time schedule constraints that were imposed before the analysis was 
initiated. The Suggestion Model DSS can be programmed to consider a large number of 
changes, and then flag the more interesting ones for management attention. Another way 
in which a suggestion model might operate is to identify the most profitable portfolios 
without constraints, and then inform the manager what outcomes could have occured if 
certain constraints and policies had been modified. 


MULTI-PURPOSE SOFTWARE SYSTEMS 


Many of the capabilities listed above can be provided simultaneously and interactively 
by some of the newer multi-purpose software packages for fourth generation computers. 
The Interactive Financial Planning System (IFPS) developed by EXECUCOMf#, for exam- 
ple, permits a number of “what if’ questions in addition to its goal structuring and risk 
analysis capabilities. Similarly, Information Builders, Inc., has developed a multi-purpose 
system called FOCUSt (For On-line Computer User), which 1s a report writing software 
package that permits applications development and data base operations. The capabilities 
of these, and many similar multi-purpose systems are listed in a recent edition of Comput- 
erWorld (PAUL[9]). Typical functions of these multi-purpose computer software programs 
are: “what if” analyses; goal-seeking; impact analyses; statistical analysis and forecasting; 
sensitivity analysis; optimization algorithms; Monte Carlo simulations; risk analysis; mar- 
keting forecasts; sales forecasts; capital equipment forecasts; manpower planning 
projections; cash flow analyses; market mix studies; and time series analyses. 

Interactive systems of this type are ideal for facilitating the involvement of higher 
echelon managers. As pointed out by PAUL[9], “what managers really need is a way to ask 
a question, get an answer, know from that answer what question they want to ask next, and 
get an answer to the second question.” This is what decision support systems are all about. 

The usefulness and versatility of these multi-purpose software systems is amazing in view 
of the short training requirements. PAUL[9] lists ten DSS software systems with a wide 
variety of analysis capabilities. Most of the training time requirements for these DSS systems 
components range from a few minutes to several days. 

The problem to be solved in building DSS for RDT&E decisions is clearly not one of 
finding candidate software systems that can be used to design the system. The difficult task 
is to integrate and modify carefully selected components from the many DSS software 


+For additional information on IFPS, contact EXECUCOM Systems Corporation, P.O. Box 9758, Austin, 
TX 78766, U.S.A. 


{For additional information on FOCUS, contact Information Builders, Inc., 1250 Broadway, New York, NY 
10001, U.S.A. 
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systems that already exist and convert these components into tailor-made systems that are 
designed to meet user needs. In performing this integrative task, it is very important that 
we not overwhelm the managers with superfluous complexity. 


ADDITIONAL SYSTEMS COMPONENTS THAT NEED TO BE INCLUDED 


The seven decision support subsystems and the multi-purpose software systems listed 
above provide a powerful set of tools for managing RDT&E portfolios. Based upon the 
writers’ experience with RDT&E organizations, however, we recommend that three 
additional systems components be included. 


(1) Distributed policy inputs and constraints 

A system is needed that would permit RDT&E executives to provide a wide variety of 
diverse inputs at one time. The information would then be automatically distributed 
wherever it was needed without requiring many separate inputs from the executive. Exam- 
ples of inputs of this type are: (a) pre-emptive decisions about the inclusion or non-inclusion 
of selected high priority efforts; (b) desired long range changes in the capabilities and 
manpower mix of the RDT&E organization that is expected to conduct the work; 
(c) program balance objectives; (d) management goals as expressed in the form of the 
annual rates at which progress is required toward the long range program balance and 
organizational capability objectives; and (e) percentage guidelines and constraints for 
portfolio mix decisions. A possible format for collecting guidelines of this type is shown in 
Table 2. 


(2) An organizational incentives index 

Another way of simplifying decisions for executives is to set up a single overall output 
index that can be used to order RDT&E proposals with respect to their expected con- 
tribution to the many goals with which the executives must be concerned. Many mathe- 
matical and quasi-mathematical solutions to the problems of assigning weights to multiple 
criterion measures have become available for this purpose in recent years, as indicated by 
the more recent conferences concerned with multiple criterion weighting problems (e.g. 
Morse[10]). An organizational incentives index based upon one or more of these methods 
would go a long way towards obtaining the cooperation of busy managers—especially if the 
managers are permitted to modify and improve the index whenever it fails to order the 
candidate RDT&E efforts in ways that are logically expected. 


Table 2. Possible format for collecting and inputting policy inputs and constraints 


Required 

Acceptable Rate of 
Policy Input Category Range Change 
Acceptable probability of -30-.95 0 
success for one effort 
Percent of RDT&E resources «10%-.20% -1% per 
allocated to Area #7 year 
Minimum orgar.izational 80 +2%, 
incentives index 
Proportion of production work 0-60% 0 
that would have to be contracted 
out 
Post RDT&E time permitted O-1 yr 0 
before product is expected 
to be profitable 
Permissible budget for $1M-$3M ¢) 
pre-emptive choices by 
R&D executives 
Proportion of RDT&E that may be 25%-75% 0 


oriented towards international 
trade 
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(3) Provocative questions from a historical perspective 

A historical perspective is useful in RDT&E plans as it is for most other decisions. 
What happened to the organization’s share of the market the last time that a similar 
product was introduced? What happened to the company’s net profits? How likely is it 
that the budget is understated? How much risk is there that the product— if developed 
as planned would be copied and marketed by one of the company’s competitors? 

Because of such considerations, a “historical analogies” subsystem is desirable. This 
subsystem should be narrative in nature, and should put more emphasis upon historically 
anchored questions of a provocative nature rather than historically-based answers. The 
amount of quantified information in the subsystem would depend upon the type of 
company being investigated, the number of similar RDT&E products developed during 
previous years, and the extent to which circumstances change from year to year in the type 
of organization for which the system was designed. It may or may not be logical to ask 
provocative questions about transistors based upon one’s experience with vacuum 
tubes—but if it is logical, some one needs to think about the issues and bring relevant 
questions to management’s attention. 


SYSTEMS OPERATION 


The three new subsystem components would be used as follows. Before interacting with 
the system, the executive would be asked to provide his own “‘policy inputs and constraints” 
using a format similar to that shown in Table 2. Then, given the components and interactive 
software subsystems described above, some interesting kinds of policy oriented feedback 
can be provided. With the aid of the file drawer subsystem the input information that has 
already been provided can be displayed. The logical implications of these inputs can then 
be summarized with the aid of the data analysis subsystem, analysis information subsystem, 
and accounting models. The implications of the manager’s input information for various 
outcomes can be estimated with the aid of the representation and optimization models. The 
various RDT&E efforts can be rank-ordered in terms of the ‘organizational incentives” 
index that the manager has created. The “‘what if” features of the multi-purpose software 
system can be used to explore other alternatives that the executive had not thought of 
before, and some interesting ideas for change in the inputs can be proposed by the 
suggestion model subsystem. In addition, the executive can be exposed to some provocative 
questions from a historical perspective based upon the company’s previous experience with 
work similar to that which is being considered. 


THE NEED TO ANALYZE DECISION MAKING REQUIREMENT IN ADVANCE 


The system described above sounds good, but there are many pitfalls to be avoided. The 
most important are: excessive complexity, overkill, and too many ‘Just in Case” (JIC) 
capabilities. Because of such risks, the first and foremost task 1s to analyze the decision- 
making requirements that exist and decide how sophisticated the policy feedback systems 
should be. If 90°, of the decisions are imposed by the market place rather than decided by 
management, the design of complex, policy-oriented feedback systems is probably a waste 
of time. THIERAUT ([8], pp. 117-146) offers some useful guidance regarding the conduct of 
feasibility studies and the kind of systems analysis that should be conducted before 
Decision Support Systems are designed. 

Another reason for looking closely at the decision making situation is that policy- 
oriented feedback systems must be justified in terms of problems that can be solved with the 
aid of such systems. A policy-oriented feedback system cannot take an inadequate market- 
ing study and make it adequate; it cannot take a poorly designed product and make it 
acceptable; and it cannot replace the really big decisions (expansion, diversification, etc.) that 
high powered executives are paid to decide. For problems that can be addressed, however, 
we can go a long way towards helping RDT&E executives to get more use out of the 
information that is available with the type of policy-oriented feedback systems that are 
described above. 
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